Skip to content

Implement SUBSTRING function pushdown for Teradata connector#37

Merged
sc250072 merged 6 commits into
developfrom
Substring_Pushdown
Sep 24, 2025
Merged

Implement SUBSTRING function pushdown for Teradata connector#37
sc250072 merged 6 commits into
developfrom
Substring_Pushdown

Conversation

@dv255037

Copy link
Copy Markdown
Collaborator

Description

This PR implements support for pushing down the SUBSTRING function to the Teradata database, improving query performance by reducing data transfer and leveraging native database string processing capabilities.

Changes Made:

  • Added RewriteSubstringFunction class to handle SUBSTRING function projection pushdown
  • Added RewriteSubstring class for expression rewriting in predicates
  • Integrated both rewriters into the TeradataClient expression and projection function rewriters
  • Enhanced function pushdown optimization framework to support string extraction operations
  • Implemented proper parameter validation and type checking for substring operations

Technical Details:

  • The implementation validates input arguments (source string, start position, optional length) before applying pushdown
  • Uses Teradata's native SUBSTRING function syntax for optimal performance
  • Supports both 2-parameter (string, start) and 3-parameter (string, start, length) variants
  • Integrates seamlessly with existing expression rewriting framework
  • Compatible with both predicate and projection contexts

Additional context and related issues

Testing Considerations:

  • Verify that SUBSTRING function works correctly with various string types
  • Validate integration with existing predicate and projection pushdown functionality
  • Test performance improvements with large datasets and complex string operations

Release notes

Teradata Connector Enhancements

New Features:

  • SUBSTRING Function Pushdown: The Teradata connector now supports pushing down SUBSTRING function calls to the database, improving query performance for string extraction operations.

Performance Improvements:

  • Reduced network overhead for queries using SUBSTRING function by leveraging native Teradata string processing
  • Enhanced execution plans for queries involving text extraction and manipulation
  • Optimized performance for analytical workloads processing large text datasets

Technical Enhancements:

  • Added support for SUBSTRING function in both predicate and projection contexts
  • Supports both 2-parameter and 3-parameter SUBSTRING variants
  • Maintains proper parameter validation to ensure safe function pushdown
  • Seamless integration with existing expression rewriting framework
  • Consistent behavior with Teradata SQL dialect requirements

Usage Examples:

-- These queries now benefit from improved performance:
SELECT customer_id, SUBSTRING(description, 1, 50) FROM products;
SELECT * FROM logs WHERE SUBSTRING(message, 1, 5) = 'ERROR';
SELECT SUBSTRING(full_name, 1, POSITION(' ' IN full_name) - 1) AS first_name FROM users;

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

```markdown
## Section
* Fix some things. ({issue}`issuenumber`)

@dv255037 dv255037 requested a review from sc250072 September 22, 2025 07:22
@dv255037 dv255037 self-assigned this Sep 22, 2025

@sc250072 sc250072 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Address the review comments.

@sc250072 sc250072 self-requested a review September 24, 2025 07:54

@sc250072 sc250072 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

@sc250072 sc250072 merged commit 62ea80a into develop Sep 24, 2025
1 of 11 checks passed
@sc250072 sc250072 deleted the Substring_Pushdown branch September 24, 2025 07:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants